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DETAILED ACTION 



Information Disclosure Statement 

1 . The reference listed on the disclosure statement (IDS) submitted on 01/1 1/2006 and 
1 1/17/2006 have being considered by the examiner (see attached PTO 1449). 



Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 1(b), which papers have 
been placed of record in the file. 



Drawings 

3. The drawings are objected to because labeled elements "110" in Figure 3a and "Set-up 
process completed. Start registering of digital video sequence" in Figure 5 have been 
mislabeled. The labeled element "110" should be changed to "310" (see [0054]) and the 
labeled element "Set-up process completed. Start registering of digital video sequence" 
should be changed to "Set-up process completed. Start recording of digital video sequence" 
(see [0062]). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate prior 
version of the sheet, even if only one figure is being amended. The figure or figure number of 
an amended drawing should not be labeled as "amended." If a drawing figure is to be 
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canceled, the appropriate figure must be removed from the replacement sheet, and where 
necessary, the remaining figures must be renumbered and appropriate changes made to the 
brief description of the several views of the drawings for consistency. Additional 
replacement sheets may be necessary to show the renumbering of the remaining figures. Each 
drawing sheet submitted after the filing date of an application must be labeled in the top 
margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the 
changes are not accepted by the examiner, the applicant will be notified and informed of any 
required corrective action in the next Office action. The objection to the drawings will not be 
held in abeyance. 

Specification 

4. Applicant is reminded of the proper language and format for an abstract of the disclosure. 

The form and legal phraseology often used in patent claims, such as "means" and "said," 
should be avoided. 

5. The specification has not been checked to the extent necessary to determine the presence of 
all possible minor errors. Applicant's cooperation is requested in correcting any errors of 
which applicant may become aware in the specification. The table below shows just a few of 
the many minor errors through the specification: 



Page no 


Line no 


Mislabeled character 


Corrected character 


15 


9 


as in Figure 4b. 


as in Figure 6b. 


16 


12 


2. Apply an image -plane 


4. Apply an image-plane 
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Claim Objections 

6. Claims 22, 26, 27, and 28 are objected to because of the following informalities: Regarding 
claim 22, this claim should depend on claim 14, not claim 9 and regarding claims 26, 27 and 
28, these claims should depend on claim 25, not claim 17. For the purpose of art rejection the 
claim 22 will be read as depend on claim 14 and claims 26, 27, and 28 will be read as depend 
on claim 25. Appropriate correction is required. 



Claim Rejections - 35 USC §102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

8. Claims 1-3, 5, 14-16, 25-27 and 36 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Aagaard (US 2003/0210329). 

Regarding claim 1, Aagaard discloses a method for generating a wide image video 
sequence, said method comprising the steps of: 

- generating a set of calibration parameters (calibration information/values) related 
to a device (multiple camera video system 1, Fig 1) having at least two video 
cameras (slave cameras in unison with the master camera) which are arranged in a 
predetermined relationship to each other (the location of each of the cameras with 
respect to each other may be determined using known techniques), said 
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parameters being unique for the at least two cameras and their current location as 
related to the object being recorded (the basic function of the calibration 
computers 240 is to calibrate the positioning of each of the remote cameras 150 
enabling the master broadcaster 2 1 5 to remotely calculate and control the slave 
cameras in response to a movement in the master camera) ([0036], [0043], [0095], 
[0107]); 

- recording synchronously video sequences using each of said at least two video 
cameras (a genlock signal may be provided to each camera 150 so that the 
resulting stored video will be synchronized across all of the camera video feeds 
during playback) ([0078]); and 

- generating a wide image video sequence from each of said synchronously 
recorded video sequences (for example, a 16:9 aspect ratio high definition camera 
with a wide angle lens could be used to capture the target image from a plurality 
of locations) ([0078] and [0172]). 

Regarding claim 2, Aagaard discloses the synchronously recorded video sequences are 
stored in a memory means (the multiple video feeds may be played back from a storage 
device, i.e. video storage devices 3a, 3b) ([0034] and [0039]). 

Regarding claim 3, Aagaard discloses the synchronously recorded video sequences are 
used concurrently for generating the wide image video sequence (a genlock signal may be 
provided to each camera 150 so that the resulting stored video will be synchronized across all 
of the camera video feeds during playback and a 16:9 aspect ratio high definition camera 
with a wide angle lens could be used to capture the target image from a plurality of locations) 
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([0078] and [0172]). 

Regarding claim 5, Aagaard discloses the wide image video sequence is stored on a 
memory means (the video images of an event of interest from the plurality of cameras will be 
stored in a suitable storage medium, such as a digital data file captured with a wide angle 
lens) ([0064] and [0172]). 

Regarding claim 14, this claim differs from claim 1 only in that the claim 1 is a method 
claim whereas claim 14 recites similar features in an apparatus format. Thus the apparatus 
claim 9 is analyzed and rejected as previously discussed with respected to claim 1 above. 

Regarding claims 15 and 16, these claims recite same limitations as claims 2 and 3, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 2 and 3 above. 

Regarding claim 25, this claim recites same limitations as claim 1 . Thus it is analyzed 
and rejected as previously discussed with respect to claim 1 above. 

Regarding claims 26 and 27, these claims recite same limitations as claims 2 and 3, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 2 and 3 above. 

Regarding claim 36, Aagaard discloses a video recording apparatus (multiple camera 
video system 1, Fig 1) comprising: 

a microprocessor (pan head computer 154 in the camera control panel 162, Fig 8), 
a memory (video storage devices 3a, 3b, Fig 1) for storing program for generating 
a set of calibration parameters (calibration information/values) related to a device 
(multiple camera video system 1, Fig 1) having at least two video cameras (slave 
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cameras in unison with the master camera) which are arranged in a predetermined 
relationship to each other (the location of each of the cameras with respect to each 
other may be determined using known techniques), said parameters being unique 
for the at least two cameras and their current location as related to the object 
being recorded (the basic function of the calibration computers 240 is to calibrate 
the positioning of each of the remote cameras 150 enabling the master broadcaster 
215 to remotely calculate and control the slave cameras in response to a 
movement in the master camera) ([0036], [0043], [0095], [0107]); 
said memory means (video storage devices 3a, 3b, Fig 1) also storing program for 
recording of wide image video sequences (for example, a 16:9 aspect ratio high 
definition camera with a wide angle lens could be used to capture the target image 
from a plurality of locations) ([0078] and [0172]).; 

read and write memory means (video storage devices 3a, 3b, Fig 1) for storing 
data relating to recorded video sequences from at least two video cameras (a 
genlock signal may be provided to each camera 150 so that the resulting stored 
video will be synchronized across all of the camera video feeds during playback) 
([0078]); and 

input means for input of manual input of parameters, input of recorded video 
sequence (he cam-A computer 228 preferably also includes a keyboard 224, 
mouse 226, or other input device which may be used locally at the master pan 
head 220), and output means for output of a wide image video sequence (there 
may also be a local monitor connected to the cam-A computer 228, and a second 
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monitor 222 may be connected to the cam-A computer 228 and reside at the 
master pan head 220) ([0099]). 



Claim Rejections - 35 USC §103 

9. The following is a quotation of 35 U.S.C. § 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claim 4 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Aagaard in view of 
Alonso (US 6,445,293). 

Regarding claim 4, the limitations of claim 1 are taught above, Aagaard does not disclose 
the wide image video sequence is transmitted live. However, Alonso discloses the wide 
image video sequence is transmitted live (the camera system sets the front camera as output 
image and transmits live video out from this camera) (3:56-65, Alonso). 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the imaging pickup device as taught by Alonso into Aagaard 's 
device, so as the camera system based on an 8-bit microcontroller and comprises four states 
of operation, which are capable to used by the main system to increase system versatility and 
can get any advanced feature shown in the main video system (3:41-45, Alonso). 

11. Claims 6-10, 17-21, and 28-32 are rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Aagaard in view of Chang (US 2004/0085451). 
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Regarding claim 6, the limitations of claim 1 are taught above, Aagaard discloses 
generating a wide image video from recorded video sequences (for example, a 16:9 aspect 
ratio high definition camera with a wide angle lens could be used to capture the target image 
from a plurality of locations) ([0078] and [0172]). Aagaard does not disclose a detailed 
calibration process according to this claim. 

However, Chang discloses the generation of calibration parameters (calibration process 
102, Fig 3) comprises the following steps: 

a. Start of calibration process (step 1 10, Fig 3) (at 1 10, a predesigned calibration 
pattern 50 is displayed in front of planar background 18, i.e. on front planar 
surface 44) ([0024], Chang); 

b. Synchronize the sequences from each camera (step 1 14 and 1 1 8, Fig 3), which 
means that at least a video sequence has to be recorded by all cameras (blocks 1 12 
and 1 16 occur simultaneously, and blocks 1 14 and 1 1 8 occur simultaneously or 
near simultaneously) ([0024], Chang); 

c. Compute inter-image projective transformations (step 122, Fig 3) (the 
correspondence mapping and geometric parameters of planar background 1 8 
determined at 120 are utilized to compute both internal and external calibration 
parameters of first camera 14 and second camera 16) ([0026], Chang); 

d. Use the transformations to refer each image to a common reference frame (step 
120, Fig 3) (at 120, CPU 30 compares the robust features from the first and 
second images to the known characteristics of calibration pattern 50 and performs 
a correspondence mapping and the correspondence mapping entails locating each 
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captured robust characteristic of calibration pattern 50 in first image 14 and noting 
the spatial relationship of the captured robust characteristics) ([0025], Chang); 

e. Choose a real or virtual reference camera (step 122, Fig 3) such that certain lines 
on the pitch and/or stadium are essentially horizontal and parallel in the image 
(the overall spatial relationship of first camera 14, second camera 16, and planar 
background 18 is used to determine a first coordinate system with respect to first 
camera 14 and a second coordinate system with respect to second camera 16) 
([0026], Chang); 

f. Select a rectangular region of interest within the image (the correspondence 
mapping and geometric parameters of planar background 18, Fig 1). This region 
contains e.g. the entire pitch and as much of the stadium as is required or visible 
(planar background 18 is determined at 124 based upon the calibration parameters 
of each camera) ([0026], Chang); and 

g. Record all computed values resulting from the calibration process to be used as 
the calibration parameters (once the two coordinate systems are derived, the 
calibration process is complete and the first subject image or video is recorded by 
first camera 14 and transferred to CPU 30 via first video capture device 32) 
([0026] and [0027], Chang). 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the calibration process as taught by Chang into Aagaard's 
device, so as the system capable to generate a synthesized image of the subject from a virtual 
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view point using the relative positioning of the first camera, the second camera, and a planar 
surface for the robust features of calibrated pattern ([0006], Chang). 

Regarding claim 7, the limitations of claims 1 and 6 are taught above, Aagaard does not 
disclose the steps of finding the lens distortion parameter(s) for each camera and correcting 
radial distortion in each image produced are comprised. However, Chang discloses the steps 
of finding the lens distortion parameter(s) for each camera, and correcting radial distortion in 
each image produced are comprised (adapted to stretch the synthesized video image on the 
video display 20 to remove any parallax distortions) ([0018], Chang). 

Regarding claim 8, the limitations of claims 1 and 6 are taught above, Chang discloses 
the step of selecting non-linear distortion parameters to reduce perspective distortion of the 
image is comprised (the internal calibration parameters include but are not limited to the 
focal length and lens distortion of each camera) ([0026], Chang). 

Regarding claim 9, the limitations of claim 1 are taught above, Chang discloses the step b 
(step 1 14 and 118, Fig 3) is performed manually by identification of corresponding features 
in concurrent video images and the coordinates for these corresponding features are input to a 
computer means (calibration pattern 50 is a predesigned pattern on a planar surface having 
robust features to be identified by CPU 30 and the size and attributes of calibration pattern 50 
have been previously entered into and are known by CPU 30; at 1 12, first camera 14 captures 
a first image of calibration pattern 50) ([0024] and Fig 3, Chang). 

Regarding claim 10, the limitations of claim 1 are taught above, Chang discloses the step 
b (step 1 14 and 118, Fig 3) is performed automatically by an algorithm for identification of 
corresponding features in concurrent video images and the coordinates for these 
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corresponding features are input to a computer means (the second image is transferred from 
second camera 16 to CPU 30 via second video capture device 34. CPU analyzes the second 
image for the robust features of calibrated pattern 50) ([0024] and Fig 3, Chang). 

Regarding claims 17, 18, 19, 20, and 21, these claims recite same limitations as claims 6, 
7, 8, 9, and 10, respectively. Thus they are analyzed and rejected as previously discussed 
with respect to claims 6, 7, 8, 9, and 10 above. 

Regarding claims 28, 29, 30, 31, and 32, these claims recite same limitations as claims 6, 
7, 8, 9, and 10, respectively. Thus they are analyzed and rejected as previously discussed 
with respect to claims 6, 7, 8, 9, and 10 above. 
12. Claims 11,13, 22, 24, 33, and 35 are rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Aagaard in view of Chang (US 2004/008545 1 ), and further in view of Matsumoto (US 
2003/0071906). 

Regarding claim 1 1 , the limitations of claim 1 are taught above, Chang discloses the 
method according to claim 1 which comprises the following steps: 

a. Apply the computed and registered calibration parameters (at 1 10, a predesigned 
calibration pattern 50 is displayed in front of planar background 18, i.e. on front 
planar surface 44) ([0024] and step 1 10, Fig 3); 

c. Obtain one new image from each camera (at 112, first camera 14 captures a first 
image of calibration pattern 50 and at 116, second camera 16 captures a second 
image of calibration pattern 50) ([0024] and steps 112 and 116, Fig 3); 

d. If required, update the parameters needed to transform intensities in one or more 
cameras to eliminate visible seams (correspondence mapping entails locating each 
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captured robust characteristic of calibration pattern) ([0025] and step 120, Fig 3); 

e. If necessary, adjust the intensities in the images from one or more cameras (a 
similar correspondence mapping procedure is completed using the second image 
to determine the geometric parameters of planar background 18 with respect to 
second camera 16) ([0025] and step 120, Fig 3); 

f. Create the current seamless, wide image from the current images from each 
camera (map corresponding robust features to determine geometric parameters of 
planar background, since the calibration pattern 50 is a planar surface located 
upon planar background 1 8, the geometric parameters of planar background with 
respect to first camera 14 are directly computed from the mapped correspondence 
information) ([0025] and step 120, Fig 3); 

g. Output the image to a display or to a memory means (storage device 37, Fig 1) 
(after all calibration parameters are determined, the overall spatial relationship of 
first camera 14, second camera 16, and planar background 18 is determined at 124 
based upon the calibration parameters of each camera and the image is to a 
display or to a memory means) ([0026]); and 

h. End of sequence (the calibration process is complete) ([0024]-[0027] and step 
110-124, Fig 3. Chang). 

Aagaard and Chang do not disclose a calibration process is repeated until the end of the 
sequence is reached or return to step b until end of generation of the video sequence. 
However, Matsumoto disclose a calibration process is repeated until the end of the sequence 
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is reached (the calibration process is repeated at a predetermined interval until the release 
switch (SW1) 126 or the release switch (SW2) 127 is pressed, Figs 7-9) ([0183], Matsumoto) 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the calibration process as taught by Chang into Aagaard's 
device, so as the system capable to generate a synthesized image of the subject from a virtual 
view point using the relative positioning of the first camera, the second camera, and a planar 
surface for the robust features of calibrated pattern ([0006], Chang). 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the calibration process as taught by Matsumoto into Aagaard 
and Chang's device, so as the imbalanced amounts and correction amounts calculated in the 
respective calibration processes are accumulated and stored in the memory since the camera 
is turned on to the time just before the exposure operation starts, and therefore, more accurate 
correction data can be obtained ([0183], Matsumoto). 

Regarding claim 13, the limitations of claims 1 and 1 1 are taught above, Chang discloses 
the new images from each video camera are read from a memory means (storage device 37, 
Fig 1) (after all calibration parameters are determined, the overall spatial relationship of first 
camera 14, second camera 16, and planar background 18 is determined at 124 based upon the 
calibration parameters of each camera and the image is to a display or to a memory means) 
([0026], Chang). 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the calibration process as taught by Chang into Aagaard's 
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device, so as the image data (e.g., video images, image shape and color information) is stored 
in storage device 37 and available for use at a later time ([0032], Chang). 

Regarding claims 22 and 24, these claims recite same limitations as claims 1 1 and 13, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 11 and 13 above. 

Regarding claims 33 and 35, these claims recite same limitations as claims 1 1 and 13, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 11 and 13 above. 
13. Claims 12, 23 and 34 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Aagaard in view of Chang (US 2004/0085451), and further in view of Alonso (US 
6,445,293). 

Regarding claim 12, the limitations of claims 1 and 1 1 are taught above, Aagaard and 
Chang do not disclose the new images from each camera are read from live sources, each 
such source comprising a video camera. However, Alonso discloses the new images from 
each camera are read from live sources, each such source comprising a video camera (the 
camera system sets the front camera as output image and transmits live video out from this 
camera) (3:56-65, Alonso). 

Thus it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the imaging pickup device as taught by Alonso into Aagaard 
and Chang's device, so as the camera system based on an 8-bit microcontroller and 
comprises four states of operation, which are capable to used by the main system to increase 
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system versatility and can get any advanced feature shown in the main video system (3:41- 
45, Alonso). 

Regarding claims 23 and 34, these claims recite same limitations as claim 12. Thus they 
are analyzed and rejected as previously discussed with respect to claim 12 above. 



Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

- Kumar et al. (US 5,963,664) discloses a system for generating three-dimensional 
mosaics from a plurality of input images representing an imaged scene; 

- Edwards et al. (US 7,373,017) discloses a system and method for capturing adjacent 
images by utilizing a panorama mode; 

Katayama et al. (US 7,098,914) provides an image synthesis method, an image 
synthesis apparatus, and a storage medium in which a mapping mode is easily set 
when synthesizing an image; and 

- Mancuso et al. (US 6,771,304) discloses a method in a digital camera for image 
capturing at least two perspective images at a constant focal length from the digital 
camera. 
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Inquiries 

15. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Sinh Tran can be reached on 571-272-7564. The fax phone number for the organization 
where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 
/TuanVHo/ 

Primary Examiner, Art Unit 2622 
KW 

3 February, 2009 



